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Abstract: In understanding how active and blended learning approaches with learning technologies engagement in 
undergraduate education, current research models tend to undermine the effect of learners' variations, particularly 
regarding their styles and approaches to learning, on intention and use of learning technologies. This study contributes to 
further examine a working model for learning outcomes in higher education with the Unified Theory of Acceptance and 
Use of Technology (UTAUT) on SRS adoption attitude, and the Study Process Questionnaire (SPQ) on students' approach to 
learning. Adopting a cross-section observational design, the current study featured an online survey incorporating items 
UTAUT and SPQ. The survey was administered to 1627 undergraduate students at a large comprehensive university in 
Hong Kong. Relationships between SRS adoption attitude, learning approaches, and learning outcomes in higher-order 
thinking & learning and collaborative learning were analyzed with a structural equation model (SEM). A total of 3 latent 
factors, including four factors from UTAUT in Performance Expectancy, Effort Expectancy, and Deep Learning Approach 
from the SPQ, were identified in the structural model on students' intention to adopt SRS in classes. Current results 
suggested that a model of active learning outcomes comprising both UTAUT constructs and deep learning approach. Model 
presented in the present study supported the UTAUT in predicting both behavioral intention and in adopting SRS in large 
classes of undergraduate education. Specifically, positive attitudes towards SRS use measured with the UTAUT, via a 
learning approach towards deep learning, accounted for variation on high-impact learning including higher-order thinking 
and collaborative learning. Results demonstrated that the process of technology adoption should be conceptualized in 
conjunction with learners' diversity for explaining variation in adoption of technologies in the higher education context. 
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1 Background 

Prevailing ideologies in higher education are advocating student-oriented, constructivist, and collaborative 
learning among students and their faculties (Al-Huneidi and Schreurs, 2013, Slavich and Zimbardo, 2012), in 
which learners must assume an active role in constructing knowledge in a collaborative and interactive 
learning environment. Active and blended learning approaches with learning technologies have been identified 
as instrumental in engaging large classes at the undergraduate level in higher education (Kerr, 2011, Exeter et 
al., 2010). Current research has proposed various models for outlining the process in which learners adopted 
such learning technologies (Venkatesh and Bala, 2008, King and He, 2006, Davis et al., 1989) and enhance their 
learning experience and performance. 

While giving feedback individually to students in large classes remains a daunting task with limited time in 
face-to-face lecture, the use of SRS allows mass, yet personalized feedback to students in the face-to-face 
learning process with aid from adopting the blended learning approach. The use of students' response system 
(SRS) with mobile devices has been demonstrated as an effective tool for engaging students in active learning 
under the large-class context in higher education (Velasco and (Javdar, 2013, Chan et al., 2013, Cobb et al., 
2010, Shapiro, 2009, Caldwell, 2007, Schell et al., 2013). 

Use of SRS for blended learning is influenced by the learner's propensity to adopt and use the technology in 
their learning practices. One of the more recent and prevailing models for understanding use of learning 
technology is the Unified Theory of Acceptance and Use of Technology (UTAUT), a comprehensive (Venkatesh 
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et al., 2003) and cross-cultural (Lidia et al., 2007) model for understanding factors to behavioral intentions and 
actual use of technology. Core components in the UTAUT include expectancies towards performance from 
adopting the technology, effort required in adopting the technology, social influence to adopt the technology, 
facilitating conditions for adopting the technology. These core components are hypothesized to influence 
satisfaction and behavioral intention to continue use of the targeted technology. 

However, these current models, when applied in an educational context, tend to undermine the effect of 
learners' variations, particularly regarding their styles and approaches to learning, on intention and use of 
learning technologies. To accommodate the moderating effect of learners' approaches, a multi-dimensional 
model is proposed for delineating the relationships between attitude towards learning technologies, learners' 
approaches to learning, and behavioral intention towards continuous use of learning technology as one of the 
strategies for practicing active learning. 

Recent research suggested that behavioral changes with SRS in classes influenced students' approaches or 
styles to learning (Tlhoaele et al., 2015, Wang et al., 2012) towards deep approach to understanding of subject 
matters. Whether such deep learning stimulated by SRS in higher education would translate into learning 
outcomes calls for a theoretical integration and an empirical investigation. 

In this context, this study contributes to further examine a working model for students' learning engagement 
with behavioral intention towards SRS usage with the Unified Theory of Acceptance and Use of Technology 
(UTAUT) (Dwivedi et al., 2011, Lidia et al., 2007, Venkatesh et al., 2003), and the Revised Two-Factor Study 
Process Questionnaire (SPQ) (Justicia et al., 2008, Biggs et al., 2001) on students' approach to learning. 

The present study is set to address the research question of whether students with high readiness towards use 
of Clickers towards active learning (UTAUT) would be more likely to adopt deep learning approach (R-SPQ-2F) 
and subsequent learners' engagement in higher order thinking and collaborative learning (NSSE). 

2 Methods 

This study features a cross-section observational study at a large comprehensive university in Hong Kong. An 
online survey incorporating items UTAUT and SPQ was administered to 1623 undergraduate students. 

Participants in this study were enrolled in undergraduate courses using students response system (SRS, a.k.a. 
Clickers) for active learning activities in classes. Details on Clickers administration in the participating courses 
were reported in a previous paper by the research team (Chan et al., 2013). 

Implementation of SRS in the current study was evaluated with an online quantitative instrument on various 
aspects of SRS implementation with 5-point Likert-type and non-Likert type questions. 

• Instruments 

• Unified Theory of Use and Acceptance of Technology (UTAUT) 

The Unified Theory of Acceptance and Use of Technology (UTAUT) augments the original technology 
acceptance model (Davis et al., 1989) with addition of social influence and facilitating conditions in the 
technology deployment environment towards measurement of technology adoption. By integrating other 
technology adoption models, the UTAUT proposed a unified model with four determinants influencing both 
behavioral intention (Bl) and actual use of a technology; these determinants are: 

• Performance expectancy (PE): perceived gains on task performance by adopting the targeted 
technology 

• Effort expectancy (EE): ease of use in adopting the targeted technology 

• Social influence (SI): degree to which significant others influence the user's intention or actual use of 
the targeted technology 

• Facilitating conditions (FC): perceived level of organizational and technical infrastructure support to 
facilitate use of the targeted technology 

Attitude towards usage of SRS as a learning tool in classroom was assessed with the Unified Theory of Use and 
Acceptance of Technology (UTAUT) (Venkatesh et al., 2003), a 5-domain 20-item instruments on a 5-point 
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likert scale measuring performance expectancy (PE), effort expectancy (EE), social influence (SI), facilitating 
conditions (FC), and satisfaction and behavioral intention to continue use of the technology adopted (S/BI). 

• Revised Two-Factor Study Process Questionnaire (R-SPQ-2F) 

Learners' learning approaches were assessed with the 20-item Revised Two-Factor Study Process 
Questionnaire (R-SPQ-2F) (Biggs et al., 2001), with learners classified as adopting surface or deep learning 
approaches for outlining their motives and strategies towards their learning. 

• Higher-Order Learning and Collaborative Learning - National Survey of Students' Engagement (NSSE- 
HO/NSSE-CL) 

Students' generic learning outcomes were captured with two subscales, higher-order learning (NSSE-HO) and 
collaborative learning (NSSE_CL) from the National Survey of Student Engagement 2013 (Indiana University 
Center for Postsecondary Research, 2013). Higher order learning refers to challenging learning tasks including 
applying information learned in solving practical problems, synthesizing information from multiple sources, 
and forming new ideas from information sourced. Collaborative learning refers to the degree that students 
master skills and concepts through discussion and team learning with other students toward formative and 
summative assessments. 

3 Data Analysis 

Quantitative data derived from this study was analyzed with a structural equation model (SEM). Data were 
analyzed using confirmatory factor analysis (CFA) and structural equation modeling (SEM) (Bollen, 1989). 
Descriptive and correlations were generated with the IBM SPSS Statistics 22 software. All SEM models were 
estimated with the IBM SPSS Amos 22 software. Standard maximum likelihood estimations were applied in the 
SEM with no observed missing data. The research model on students 1 learning outcomes was based on two 
latent variables, namely satisfaction and behavioral intention to use of SRS in class in the Unified Theory of Use 
and Acceptance of Technology (UTAUT), and students 1 deep approach to learning with the Revised Study 
Process Questionnaire - 2 Factor Structure (R-SPQ-2F). The research model is illustrated in Figure 1 and the 
research hypotheses are listed in Table 1. 



Table 1: Research hypotheses 
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HI 

Performance expectancy (PE) will have a positive and significant influence on satisfaction and 
behavioral intention to use SRS (S/BI) 

H2 

Effort expectancy (EE) will have a positive and significant influence on satisfaction and behavioral 
intention to use SRS (S/BI) 

H3 

Social influence (SI) will have a positive and significant influence on satisfaction and behavioral 
intention to use SRS (S/BI) 

H4 

Facilitating conditions (FC) will have a positive and significant influence on satisfaction and behavioral 
intention to use SRS (S/BI) 

H5 

Satisfaction and behavioral intention to use SRS (S/BI) will have a positive and significant influence on 
deep learning approach (Deep) 

H6 

Deep learning approach (Deep) will have a positive and significant influence on NSSE Higher-Order 
Thinking / Learning (NSSE-HO) 

H7 

Deep learning approach (Deep) will have a positive and significant influence on NSSE Collaborative 
Learning (NSSE-CL) 


The proposed model fit was evaluated using conventional fit indices (Hu and Bentler, 1999) including Chi- 
2 

Square Statistics (x ), global fit index (GFI), Tucker-Lewis index (TLI), comparative fit index (CFI), incremental fit 
index (IFI), root mean square error of approximation (RMSEA), and the standardized root mean square 
residuals (SRMRs). 

4 Results 

4.1 Participants and demographics 

A total of 1623 undergraduate students from 39 courses at a 4-year university participated in this study. The 
mean age of participating students is 18.93 year old, with 30.44% of the study sample being freshmen (year-1 
students) at age 17 or below, 67.1% being sophomores between age 18-23 or below, and 2.46% being mature 
students age 24 or above. Figure 2 illustrated the distribution of participants' age. 


24-Ye*r-Otd or Above n=40 2 SX 



Figure 2: Age of participating students 
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Students in this sample represented a balanced mix of academic disciplines, with business and health / social 
sciences students accounting for 60% of the study population while science / technology / engineering (STE) 
students (19%), tourism & hospitality students (16%), and humanities / design students (5%) rounded up the 
remaining 40% of the study sample. Figure 3 depicted distribution of students' disciplines. 



Figure 3: Disciplines of participating students 


Structural model and path coefficients of learning technology use, deep learning, and learning outcomes 



Figure 4: Structural Equation Model of UTAUT constructs, behavioral intention and satisfaction towards 
learning technology and R-SPQ-2F deep learning approach on NSSE higher-order learning and collaborative 
learning 
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The proposed structural model is illustrated in Figure 4, where Table 2 presents correlation among model 
variables and Table 3 highlights the critical ratios of exogenous (predictor) variables included in the research 
model. Table 4 presents the path coefficients and corresponding statistical significance for each of the 
hypotheses. 

Upon fitting the study data, results of the proposed research model exhibited a good fit: (x2 = 3126.104, df = 
682, x2/df = 4.584, GFI =0.891, TLI = 0.934, CFI = 0.939, IFI = 0.939, RMSEA = 0.047, SRMR= 0.038). Overall, five 
out of seven hypotheses were supported by the data. 

Two out of four hypotheses (HI and H2) representing the relationship among the main UTAUT constructs (PE, 
EE) to S/BI were supported in this study. As shown in Table 4, performance expectancy (PE) positively 
predicted satisfaction and behavioral intention (0.415, p < 0.001); therefore, HI was supported. Similarly, 
effort expectancy (EE) significantly predicted satisfaction and behavioral intention (0.258, p < 0.001); 
therefore, H2 was supported. 

The hypotheses that were not supported was H3: SI to S/BI and H4: FC to S/BI. Social Influence (SI) (2.644, 
p=.008) and Facilitating Condition (FC) (1.630, p=.103) did not significantly predict satisfaction and behavioral 
intention to continual use of SRS; therefore, H3 and H4 were not supported. 

Satisfaction and behavioral intention to continue use of SRS (S/BI) significantly predict deep learning approach 
(Deep) (0.287, p < 0.001); therefore, H5 was supported. 

Regarding learning outcomes, deep learning approach (Deep) positively predicted usage higher-order thinking 
and learning (NSSE-FIO) (0.523, p < 0.001) and collaborative learning (CL) (0.153, p < 0.001); thus H6 and H7 
were supported. 


Table 2: Correlation coefficient of model variables 



EE 

SI 

FC 

Satisfaction 

Deep 

HO 

CL 

Age 

PE 

.690*' 

.729** 

.676** 

.783** 

.468** 

.226** 

.184** 

-.134** 

EE 


.637** 

.744** 

.738** 

.387** 

.254** 

.182** 

-.103** 

SI 



.711 

.713 

.463** 

.239** 

.223 

-.083** 

FC 




.726** 

.421** 

.307** 

.240** 

-.100** 

Satisfaction 





.420** 

.231** 

.182** 

-.117** 

Deep 






.386 

.283 

-.053* 

HO 







.428 

-.017 

CL 








.037 

PE: Perceived ease of use; EE: Effort Expectance; SI: Social influence; FC: Facilitating Conditions; 
Deep: Deep approach; HO: Higher Order Thinking; CL: Collaborative Learning; **p < 0.01; *p <0.05 
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Table 3: Standard error and critical ratios for each of the parameters 



Estimate 

S.E. 

C.R. 

P 

Label 

PE 

0.452 

0.023 

20.034 

< .001 

par 52 

EE 

0.531 

0.024 

21.744 

< .001 

par 53 

SI 

0.405 

0.02 

19.937 

< .001 

par 54 

FC 

0.316 

0.019 

16.922 

< .001 

par 55 

Age 

6.714 

0.236 

28.478 

< .001 

par_56 


Table 4: Structural equation model path coefficient results 



Path (Hypothesis) 

Standardized path coefficient 
(Beta) 

t-value 

Hypothesis testing result 

HI 

S/BI <- PE 

0.415 

8.839 *** 

Supported 

H2 

S/BI <~ EE 

0.258 

6.555*** 

Supported 

H3 

S/BI <~ SI 

0.249 

2.644n.s. 

Not supported 

H4 

S/BI <- FC 

0.16 

1.630 n.s. 

Not supported 

H5 

S/BI <-- Deep 

0.287 

16.018*** 

Supported 

H6 

NSSE-HO <~ Deep 

0.523 

14.466*** 

Supported 

H7 

NSSE-CL <— Deep 

0.153 

5.014*** 

Supported 

Model fit indices: x2 = 3128.587, df = 683, x2/df = 4.581, GFI =0. 891, TLI = 0. 934, CFI = 0. 939, IFI = 0. 939, 
RMSEA = 0. 047, SRMR= 0. 038; ***p < 0.001; n.s. Not significant. 


5 Discussion and implications 

A model of 3 latent factors, including two factors from UTAUT in Performance Expectancy and Effort 
Expectancy along with Deep Learning Approach from the SPQ, were identified in the structural model on 
students' learning outcomes upon adopting SRS in classes. 

Hypotheses HI & H2 were statistically significant, indicating evidence for substantial influence on behavioral 
intention to adopt education technology based on perceived performance and effort expectancies. 
Particularly, the observed effect size of performance expectancy with standardized coefficient of 0.415 in the 
research model was substantially larger than average coefficient estimated in a recent meta-analysis on 
UTAUT of 0.343 (Dwivedi et al., 2011). 

Performance expectancy may play a significant role in adoption of education technology towards learning 
goals in the Asian culture. It has been suggested that Asian, particularly Chinese students, tend to value the 
avoidance of poor performance and extrinsic motivation (D'Lima et al., 2014) high in formulating their learning 
goal orientation. If adoption of certain educational technology could significantly improve on their 
performance and avoid failure, it is likely that students would value performance expectancy in making 
decision to adopt technology for learning over other factors. 

Hypotheses H3 & H4 did not reach statistical significance in the current research model. While Social Influence 
(SI) and Facilitating Conditions (FC) did not predict satisfaction and behavioral intention, the overall model 
nonetheless suggested linkage between readiness towards SRS and tendency to engage in deep learning, 
which in turn positively predict higher-order thinking and learning and collaborative learning with peers. 


www.eiel.org 


437 


ISSN 1479-439X 



























The Electronic Journal of e-Learning Volume 13 Issue 6 2015 


Though regarded as one of the more significant constructs in UTAUT, the contribution of SI is subjected to 
further investigations since it is sometimes omitted in systematic review and meta-analysis because of its 
context and cultural specificity (Lidia et al., 2007). Indeed, non-significant path between SI and other model 
variables parallels findings about the inconsistent relationship of this construct with the overall model in the 
technology adoption literature (Lee et al., 2003) in general. Exclusion of SI in the UTAUT-based research model 
has also been resonated in a similar non-significant finding from a study population of Asian origin (Koh et al., 
2010). Further investigation of cross-cultural variation of this dimension is required to delineate the role of 
social influence in education technology adoption in the Asian context, particularly when level of individualism 
and avoidance of uncertainty that vary significantly across cultures (Nistor et al., 2014). 

In the higher education context, findings on the role of facilitating conditions (FC) in UTAUT have been mixed - 
from key correlates to effort expectancy (Terzis and Economides, 2011)] to non-significant factor excluded 
from the research model (Al-Hujran et al., 2014). A possible explanation to the non-significant path observed 
in the current study is that the participating classes using SRS were part of an ongoing e-learning project that 
provides centralized support and training on developing and delivering SRS questions in classes. Further 
analyses are required to examine whether uniformity in technical and user support explains the lack of 
variation in terms of facilitating conditions towards technology adoption. 

Hypotheses 5-7 were supported in the research model presented. Findings from the current study confirms 
the effect of SRS on encouraging complex cognitive effort (Brady et al., 2013a) and collaborative learning 
(Jones et al., 2012, Blasco-Arcas et al., 2013), while bridging these learning outcomes via activation of deep 
learning approach being facilitated by satisfaction and intention to use SRS. Soliciting students' responses in 
learning environment has been suggested to facilitate retrieval-based learning (Karpicke and Grimaldi, 2012, 
Campbell and Mayer, 2009) Through retrieval of recently acquired information, students consolidate their 
acquired information in the higher education setting with active learning effort and timely feedback (Lantz and 
Stawiski, 2014). The student-initiated effort in SRS also resonates with the generation effect (Hirshman and 
Bjork, 1988, Lantz, 2010), a theory in cognitive and learning psychology hypothesizing better memorization 
and retention in learning when individuals are required to generate learning artifacts rather than simply 
receiving and encoding information. These cognitive processes in a learning context possibly account for the 
demonstrated association between deep learning approach and readiness to SRS demonstrated in this study. 

The presented research model highlights the long-term impact of using SRS in classes beyond immediate gains 
such as students' attention and class engagement (Velasco and (javdar, 2013, Vaterlaus et al., 2012, Kay and 
LeSage, 2009, Morling et al., 2008). Appropriate and effective use of SRS should be regarded as an integral 
mean to achieve deep learning strategies towards enriched, applied, and collaborative learning, which are 
considered high-impact practices in higher education in the 21 st century (Kilgo et al., 2014). 

During SRS session, the level of students monitoring their cognitive effort could vary substantially, leading to 
corresponding variation in learning outcome through SRS. Recent studies suggested that SRS questions and 
subsequent students' responses effect moderately on heightening students' metacognitive effort by verifying 
their knowledge mastery with the responses presented and discussed in classes (Brady et al., 2013b, Brady et 
al., 2013a);Schell, 2013 #1066]. The current model could possibly address students' varying level of 
metacognitive activities during SRS sessions with deep learning approach, an orientation calling for 
metacognition and mastery of knowledge acquisition, partially contributing to students' motivation and effort 
to think through the questions presented towards a reasoned response. 

5.1 Limitations of the current study 

Learning approach and attitude towards acceptance and use of technology are limited and subjected to recall 
and social desirability biases by self-reported items from survey. In delineating the moderating effect of 
learning approach between intention and behavior, further inquiries through qualitative data in a mixed 
method design would provide more accurate explanations on the findings currently presented in this paper. 

Implication for learning with technology in higher education: 

Findings from the present study suggested that using SRS, when students are properly motivated with its 
utility on learning performance, could influence deep learning approach that in turn predicts higher-order 
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learning activities such as higher-order thinking and collaborative learning. Teachers using SRS to facilitate 
deep learning could focus on SRS questions that are deemed fitting to such learning approach, such as critical 
thinking or application questions (Bruff, 2009) that optimize stimulations in their students towards higher- 
order learning. 
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